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Executive Summary 



I'll 



S ince 2002 the Consortium on Chicago School Research, in collaboration with the Mills College 
in Oakland, California, has conducted a series of studies on the Chicago High School Redesign 
Initiative (CHSRI) — a partnership between Chicago Public Schools (CPS), the Gates Foundation, and 
local Chicago foundations to create approximately two dozen small high schools across the city. 

This latest report in the CHSRI series describes the practices and characteristics of CHSRI schools 
with better than expected freshman-year course performance. Drawing on both qualitative and quan- 
titative indicators across a sample of ten CHSRI schools, our analysis identified three conditions found 
in schools with comparatively high student achievement: strong teacher professional communities, deep 
principal leadership, and strong teacher influence. In addition, CHSRI schools with high achievement 
tended to provide a personalized and supportive environment for their students. 

Through qualitative fieldwork, we also show how these characteristics work together in high achiev- 
ing schools. Teacher professional communities that engage in collective work on academic improvement 
are supported by strong leadership in schools. Involved principal leadership is important for organizing 
and sustaining collective work, while teacher influence helps make it more relevant for schools’ staff. 
Both of these supporting conditions are necessary to make collective work more meaningful. 

This work highlights that how adults work together in small schools is a crucial factor in rais- 
ing student achievement. In particular, it suggests that collective work on improving instruction is a 
key lever for raising achievement. In addition, it points to the benefits of balancing the direction and 
initiative provided by principals with teacher voice and leadership. Given that reducing size does not 
automatically lead to such developments, however, schools will need to intentionally focus on creating 
these key organizational characteristics. 



If Small Is Not Enough 





Introduction 






M aple High School and Wood Grove Academy have much in common. As small high schools 
created under the Chicago High School Redesign Initiative, both value creating personalized 
relationships for students and a collegial environment for teachers. Both also have school themes to 
guide curriculum development. Despite these similarities, however, the two schools differ in important 
ways. Their efforts to facilitate teacher collaboration illustrate one crucial difference. At Maple High 
the principal suggested to his staff that they create a model for peer classroom observations. He worked 
closely with teachers to develop the initial model and monitored its implementation. As the principal 
tracked its progress, he noted two challenges: teachers did not always give constructive feedback for 
fear of saying “mean things to each other,” and they often had conflicting instructional philosophies. 
In response to these problems, the principal decided to add clear and explicit expectations of good in- 
struction to the school’s peer observation model. He gathered ideas about what those elements might 
be and brought them to the school’s staff for discussion. Over several sessions of in-house professional 
development led by the principal, the school’s staff identified the model of instructional practice they 
wanted to embrace for peer observation. 

Wood Grove Academy’s principal also wanted to develop teacher collaboration. According to the 
school’s teachers, she created an atmosphere where they felt encouraged to observe peers’ classrooms 
and share instructional practices. Despite this encouragement, collaborative practices existed, as one 
teacher described it, “only on paper,” since no one monitored if or how teachers worked together. 
Furthermore, teachers reported that collective improvement activities were unfocused and often pulled 
people in several directions. In short, Maple High and Wood Grove Academy differed from each other 
on school characteristics such as principal leadership and professional community. In the first case of 
Maple High, the principal was actively involved in initiating and participating in collective improve- 
ment work. By contrast, the principal of Wood Grove Academy provided little direction for how to 
increase collaboration and teachers rarely worked together on sustained projects. 

As urban districts increasingly turn to small school initiatives to address the persistent problems 
confronting high schools, reformers must consider how differences like those found in Maple High and 
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Wood Grove Academy are related to small schools’ 
potential to improve student achievement. 1 Recent 
accounts of the impact of school size are mixed: 
while some studies find evidence supporting 
claims that small schools lead to better outcomes 
for students, others find that positive outcomes in 
small schools are inconsistent at best. 2 Are school 
characteristics like those found in Maple High and 
Wood Grove Academy related to differences in 
student achievement? If so, which characteristics 
are associated with high achievement? This report 
attempts to provide some initial answers to these 
questions through a study of the Chicago High 
School Redesign Initiative (CHSRI). Started 
in 2001, CHSRI aimed to open more than two 
dozen small high schools across Chicago. The 
Consortium on Chicago School Research at the 
University of Chicago, in partnership with Mills 
College in Oakland, California, has undertaken a 
series of qualitative and quantitative studies of the 
initiative. In this report we examine the character- 
istics and practices of CHSRI schools with strong 
student achievement. We begin below by discuss- 
ing previous research on what makes small schools 
successful. We then provide additional background 
information on the CHSRI initiative and describe 
how we conducted our study. Finally, we present 
our findings and discuss how they may inform the 
work of small school reformers and researchers. 

What Makes Small Schools Successful? 
The unofficial motto of the small schools move- 
ment may be “Small is not enough.” 3 Reformers 
often caution that reducing school size is no pana- 



cea for the challenges facing schools, but is only a 
lever to help facilitate positive changes in school 
environment, teaching, and student learning. 
Both research and anecdotal observations seem 
to corroborate this warning: for every example of 
a small school successfully creating a supportive 
and enriching learning environment for students 
(e.g., Frederick Douglass Academy in New York 
City), there are equally dramatic examples of small 
school experiments (e.g., the recently closed and 
much discussed Manual High School in Denver, 
Colorado) that did little to improve the educational 
experiences of students. 4 Despite such inconsis- 
tencies, there has been enough positive evidence 
to persuade several districts across the country to 
implement large-scale small school initiatives. As 
they do so, districts are trying to create schools that 
not only are small but also that have organizational 
characteristics that are associated with high student 
performance. This reality makes it necessary to 
broaden the discussion beyond the impact of size 
on school performance and begin to identify the 
conditions that are necessary to create small schools 
with strong student outcomes. 

While there has been little systematic research 
addressing this question specifically, a number of 
researchers and reformers have identified several 
school practices and characteristics that may help 
explain why some small schools are more success- 
ful than others at bolstering student achievement. 
First, establishing school environments that pro- 
vide personalized academic and social support to 
students may be one key to improved learning. 
Darling-Hammond et al. observed that successful 
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small schools create strong relationships between 
students and teachers as well as structures to 
support them . 5 Several other studies also found 
a relationship between personalized school envi- 
ronments and student achievement . 6 In contrast, 
small schools may see fewer benefits when they do 
not create these structures to facilitate supportive 
relationships and contexts . 7 

Developing strong teacher professional com- 
munities focused on improving instruction and 
student learning also may be important for suc- 
cess. Previous studies have shown that teachers in 
successful small schools are given adequate time to 
plan together and make productive use of this time 
by critiquing and improving each other’s instruc- 
tional practices . 8 In addition, successful schools of- 
ten provide teachers with several opportunities for 
professional growth and development within the 
school . 9 Supovitiz and Christman found that when 
teachers in small learning communities engaged in 
structured and sustained discussions about instruc- 
tion, student achievement improved . 10 

Another important aspect of success appears 
to be school leadership. To meet the demands 
brought about by school restructuring and the in- 
creased workload associated with having a smaller 
staff, several studies argue that small schools need 
to distribute leadership and allow teachers to 
play an active role in decision making and school 
management . 11 Traditional top-down leadership 
undermines school morale, trust, and profession- 
alism, and may have adverse affects on student 
outcomes . 12 Yet without good principal leadership, 
small schools risk not developing key school struc- 



tures and supports or having them decline over 
time . 13 Strong principal leadership also is important 
for establishing a vision for school improvement 
and efforts to implement it. In earlier research on 
CHSRI small schools, we found that principals 
were crucial catalysts in helping teacher communi- 
ties engage in structured and sustained collective 
work on instructional improvement. Without prin- 
cipal leadership in this area, teachers were unlikely 
to organize these efforts on their own . 14 

The most important element of success, 
however, may be schools’ emphasis on pushing 
all students towards high academic achievement. 
Referred to as academic press, this collective focus 
on rigorous academic work often is cited as the 
necessary element to move small schools beyond 
being simply nurturing environments to places 
where students can develop their full intellectual 
potential and promise . 15 

While all these factors are believed to be 
important for the success of both large and small 
schools, there is little understanding of how they 
work across these different contexts. Furthermore, 
there are few studies examining the association of 
these factors with student outcomes in major small 
school initiatives like CHSRI. Finally, it is not clear 
whether these factors can produce strong student 
outcomes in isolation or if they need to work in 
combination with one another. And if the latter 
is the case, which specific combinations of factors 
should be present in small schools? In this report 
we attempt to answer these questions by identifying 
CHSRI small schools with high levels of student 
achievement given their student populations and 



CONSORTIUM ON CHICAGO SCHOOL RESEARCH AT THE UNIVERSITY OF CHICAGO 



5 



TABLE 1 

Performance and Demographic Characteristics of First-Time Freshmen, 2005-06 





CHSRI Schools 


All Other 
High Schools 


All Other 
Non-Selective 
High Schools 


Incoming Eighth-Grade ITBS Reading 


230 


246 


240 


Racial Composition Percentage 








African American 


85 


49 


51 


Latino 


13 


38 


39 


White 


2 


9 


7 


Percentage Receiving 
Special Education Services 


23 


16 


17 


Percentage Old for Grade 


35 


25 


28 


Percentage with Two or More Moves in Three 
Years Prior to High School 


14 


8 


9 



The national average Iowa Tests of Basic Skills (ITBS) score for an eighth-grader is 250, for a seventh-grader is 239, and for a sixth-grader is 227. 



comparing their characteristics to other CHSRI 
schools with lower levels of achievement. 16 

A Brief Description of the 
CHSRI Initiative 

The CHSRI initiative is a collaborative reform ef- 
fort between several funders and Chicago Public 
Schools (CPS). With an initial $12 million grant 
from the Bill & Melinda Gates Foundation and 
$6 million in matching funds from local funders, 
its goal is to create approximately two dozen small 
high schools across the city. 

The initial strategy of the initiative was to 
close large underperforming high schools and 
convert them to several small autonomous schools 
within the same building. This process began in 
2002 as CHSRI opened five small high schools 



on three large high school campuses that were 
scheduled to phase out over the next three years. 
At the start of the 2003 academic year, four ad- 
ditional schools were opened on these campuses, 
and a final set of three opened in 2004. 

With a grant of approximately $ 8 million dol- 
lars in 2003, CHSRI expanded its strategy to create 
new start (not converted) high schools — schools 
with completely new cohorts of students and teach- 
ers. In 2004 the first wave of these new CHSRI 
schools began with the creation of two schools 
connected to the Big Picture Schools network. 
In 2005 two additional new starts were created. 
And finally, in 2006 an additional seven new start 
schools were created on two campuses. 

Since the inception of the initiative, CHSRI 
schools served predominately African American 
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and Latino students, most of whom have low aca- 
demic achievement and high academic needs. This 
is especially true of the freshman cohorts in schools 
opened prior to 2006, the group from which our 
study’s sample is drawn. 17 As illustrated by Table 1, 
the 2005—06 freshman cohorts from these schools 
were predominately African American. Their aver- 
age eighth-grade Iowa Tests of Basic Skills (ITBS) 
score in reading was 230, approximately two years 
below grade level. In addition, a little less than 
one -fourth of these CHSRI students needed spe- 
cial education services, compared to 16 percent in 
other district high schools. More than one-third 
of the students had been held back at least a year 
in elementary school and thus were old for their 
grade. In sum, CHSRI schools serve students most 
in need of a high-quality educational experience. 

How We Conducted the Study 
To identify the characteristics and practices of 
CHSRI schools with strong student outcomes, we 
draw on a combination of quantitative and quali- 
tative data in a sample of ten schools. Each school 
in our sample participated in the Consortium’s 
biannual district-wide survey administered in 
April and May of 2005. In addition, we conducted 
principal interviews and one teacher focus group 
in each school during May and June of 2006. The 
general strategy for our analysis was to identify 
schools in our sample with strong student achieve- 
ment and compare them with other schools in the 
sample on factors potentially related to student 
achievement. 



Indicator of Student Achievement 

We identified schools with strong student achieve- 
ment using records of first-time freshman grades 
for the 2005-06 school year. Specifically, we 
examined schools’ average GPAs and on-track (to 
graduate) rates. Students’ unweighted GPA was 
used for the analysis where four points are given 
for an A, three for a B, two for a C, one for a D, 
and none for an F. A school’s on-track (to graduate) 
rate is the proportion of students who have accu- 
mulated five credits and have no more than one 
failing grade in a semester course in a core subject 
(English, math, science, and social science) by the 
end of the freshman year. 18 

Some may argue that indicators based on 
course performance are inappropriate because 
grades are subjective, easily manipulated by 
schools, and not as predictive as standardized tests. 
We contend, however, that grades offer a valid 
and strong indication of students’ achievement. In 
Chicago there is little evidence of grade inflation 
in schools. On the contrary, it appears to be rather 
difficult for students to get A averages. Overall 59 
percent of 2002 and 2003 graduates completed 
school with a 2.5 GPA or less, while only 7 percent 
graduated with a 3.5 or higher average. Fewer than 
20 percent of Latino and African American stu- 
dents graduated with a GPA higher than 3.0. More 
importantly, these trends do not vary much by 
school. After controlling for students’ background 
characteristics, differences in GPA and the number 
of Fs students receive between the highest and 
lowest performing schools is modest — 0.3 grade 
points and 1.4 Fs respectively. Furthermore, stu- 
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dents with low incoming test scores do not receive 
better grades at low performing schools. 19 

In addition, students’ performance in courses, 
independent of test scores, is a strong indicator 
of student achievement. Allensworth and Easton 
show that freshmen who are on-track (to graduate) 
are four times more likely to graduate high school 
than students off-track, and that on-track status is 
a stronger predictor of graduation than incoming 
standardized test scores. 20 In addition, Roderick, 
Nagaoka, and Allensworth find that grades are 
crucial for success beyond high school. Specifically, 
they demonstrate that grades matter more than 
standardized test scores for college access and 
are the best predictor of students’ likelihood of 
college graduation. 21 Thus, both grades and on- 
track status are important indicators of academic 
achievement and are related to students’ overall life 
chances and opportunities. 

Finally, we chose to not use standardized test 
scores as an indicator of student achievement for 
several reasons. First, in Chicago it is difficult to 
determine with confidence which schools have 
better average standardized tests scores, even 
after controlling for school demographics. This 
is because some schools retain relatively more of 
their low-performing students and have them take 
standardized tests during their sophomore and ju- 
nior years. Such schools will incorrectly look worse 
compared to other schools since their average scores 
reflect a wider range of students. Even when accu- 
rate comparisons of schools’ average test scores are 
available, the frequent lack of rigorous alignment 
between standardized tests and state learning stan- 



dards suggests that tests may be poor indicators of 
the content and quality of classroom instruction. 22 
In fact, in schools where more teachers mold their 
instruction and curriculum to improve test scores, 
standardized tests may become better indicators of 
a school’s focus on test preparation than their suc- 
cess in fostering authentic student achievement. 2;> 
Given all of the above considerations, we believe 
that course performance offers a more useful and 
valid measure of student achievement. 

To create an indicator of student achievement, 
we compared the performance of each school in 
our sample to the average performance of all high 
schools in the district serving a similar student 
population. CHSRI schools in our sample were 
coded as having strong course performance if both 
their average GPA and on-track rate were statisti- 
cally better than the district average. 24 Appendix 
C provides a complete description of the model 
and variables we used for this analysis. 

Indicators of School Characteristics Potentially 
Important for Student Achievement 

Indicators of school characteristics come from a 
combination of CCSR surveys and qualitative 
fieldwork. Teacher and student survey descrip- 
tions are based on responses from 3,273 teachers 
in 52 schools and 15,929 first-time freshmen in 71 
schools. Our CHSRI sample draws on 220 teachers 
and 753 students in ten schools. We constructed 
survey measures indicating the degree to which 
teacher leadership, academic press, and student 
supports exist in schools. Appendix A contains 
details about the analytic samples, and Appendix 
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B describes the Rasch and principal components 
analysis methodology used to create survey mea- 
sures and factors. 

We also draw on fieldwork conducted in all 
20 CHSRI schools, but primarily report here on 
data from our sample of ten CHSRI schools that 
also have complete student and teacher survey data. 
In each school we conducted principal interviews 
and teacher focus groups, talking with a total of 
56 school staff across our sample schools. From 
this data we were able to evaluate both principal 
leadership and teacher professional communities 
in schools. See Appendix D for a description of 
our qualitative analysis. 

We coded schools in our sample for whether 
they were strong in each of these areas of interest. 
For quantitative indicators this was done by using 
hierarchical models to identify schools with teacher 
and student survey responses that were statisti- 
cally more positive than schools serving similar 
students. 23 A complete description of the models 
and variables used can be found in Appendix C. 
For each qualitative indicator we used specific 
criteria, based on the concepts we were trying to 
capture, to identify strong schools. These defini- 
tions will be described below in our discussion of 
each indicator. 

The indicator of teacher professional com- 
munity comes from qualitative data. During our 
fieldwork we asked principals and teachers to iden- 
tify the activities they pursued during the year to 
improve academically. We used their responses to 
examine whether schools engaged in developmen- 
tal practices — regular group interactions focused 



on common instructional issues — to pursue their 
improvement goals. 26 To do this we coded each 
improvement activity in each school for whether 
it involved a developmental component. Activities 
received this designation if they were not related 
to any external standardized test preparation ac- 
tivities and met three criteria. First, the improve- 
ment activity had to have a focus on a specific and 
shared instructional issue. Some examples of these 
activities from our fieldwork included aligning 
curricula to state learning standards, improving 
student assessments, implementing differentiated 
instruction, and developing outcome-based learn- 
ing goals. Peer classroom observations activities 
met this criterion if feedback to teachers was based 
on specific, explicitly shared instructional norms 
or expectations. Second, the activity had to in- 
volve teachers in group work or discussion about 
the common issue. The group work could occur 
in teacher meetings or professional development. 
And finally, the collaborative component of the 
activity had to be sustained over time, indicated 
by references to either regular (e.g., once every two 
weeks) or less frequent meetings, but consistent 
meetings over a long period of time. Schools with 
at least one improvement activity that met these 
requirements were coded as having a strong teacher 
professional community. 

During our fieldwork we also examined the 
role principals play in improvement efforts. We 
noted several different tasks principals perform in 
relation to improvement activities such as, but not 
limited to: identifying problems to be addressed; 
developing improvement strategies; coordinat- 
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ing and participating in improvement work; and 
monitoring the progress of activities. In addition, 
we observed that some principals engage in what 
we call deep principal instructional leadership — 
or intensive involvement in specific improvement 
activities marked by the performance of multiple 
leadership tasks. Principals were coded as exhibit- 
ing deep leadership if they performed at least two 
concrete, long-term or central tasks associated 
with an instructional improvement activity. For 
example, one principal wanted to improve the 
instructional and relational skills of his teachers 
for meeting the needs of low-income students. 
After identifying this as an issue to be addressed, 
he organized a reading group in which all teachers 
were required to participate. He and the teachers 
used the group to develop instructional strategies 
for groups of students in their school. The principal 
also organized a retreat with the author of the book 
they read to refine their improvement strategies. 
After the retreat he worked with teachers in the 
school to implement what they had developed. In 
sum, this principal performed several tasks that 
went beyond transitory involvement and were 
necessary for the initiation, implementation, and 
continuation of the activity. 

Our indicator of teacher leadership is a 
survey measure capturing teacher influence in 
schools. It asks teachers how much influence they 
have over school policy in several areas such as 
hiring, school scheduling, budgets, professional 
development, and instruction. It also asks more 
generally how involved teachers are in important 
decision making and if they feel comfortable 



voicing their concerns. 

For our indicator of personalized student sup- 
ports we created a factor that combines six survey 
measures. The first measure in the factor, School- 
Wide Future Orientation, asks students how much 
teachers work to make sure that all students stay 
and succeed in school as well as help students plan 
for their future. Sense of Belonging asks students to 
report their feelings of fitting in at school, whether 
school feels like a family, and the level of their par- 
ticipation in school activities. It also asks whether 
students have people in their school who care about 
and help them. Classroom Personalism measures 
whether students’ math or English teacher gives 
them individual assistance with their academic 
problems. Through the Student-Teacher Trust mea- 
sure, students report whether teachers care about 
them, keep their promises, try to be fair, listen to 
students’ ideas, and treat students with respect. 
Our measure of Teacher Support asks students if 
there is at least one teacher who cares about how 
they are doing, would be willing to help with a 
personal problem, and would talk with them if 
they were having problems in class. Finally, Peer 
Support for Academic Achievement measures how 
much students talk about what they did in class, if 
they help each other with homework, and if their 
friends think it is important to attend class. 

We use a survey measure of academic press 
to indicate the degree to which students are chal- 
lenged academically in their schools. It includes 
items asking whether teachers expect all students 
to work hard and the degree to which their work 
is challenging. 
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In the final step of our analysis we organized 
quantitative and qualitative descriptions of each 
school into a data table to identify combinations of 
school characteristics that produce strong student 



achievement. Appendix D details the method 
used for our analysis. The next section presents 
the results. 



Introduction Endnotes 



1 Other cities that have implemented small schools initiatives include 
New York, San Diego, and Los Angeles. See Allensworth (2005); 
Ponisciak (2005); Green and Winters (2005); Allensworth (2004); 
Powell, Farrar, and Cohen (1985) for examples of problems 

in urban schools. 

2 See Lee (2002); Holland (2002); Mitchell (2000) for studies 
showing small schools having a positive impact and Kahne, Sporte, 
and de la Torre (2006); Mitchell, et al. (2005); Hess and Cytrynbaum 
(2002); Wasley et al. (2000) for examples of an inconsistent impact. 
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5 Darling-Hammond et al. (2002). 
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12 Oxley (1994); Copland and Boatright (2004). 

13 Shear et al. (2005). 

14 Stevens (2006). 

15 Mohr (2000); Wasley et al. (2000). 

1 6 Other factors such as autonomy, accountability, and initial 
planning process may also be related to success. While there are 
some differences in these areas across CHSRI schools depending on 
when they were started, and these differences could shape schools’ 
ability to develop certain organizational characteristics, we do not 
believe they change how organizational characteristics influence 
student achievement. Furthermore, the impact of these differences is 
minimized since CHSRI schools operate under the same set of district 
guidelines and began with comparable levels of financial and technical 
support. Finally, any differences in these areas do not appear to be 
critical to student outcomes as the high performing schools identified 
in this study come from several different cohorts of CHSRI schools. 
Thus, we can focus our attention on the relationship between key 
school level conditions and student achievement. 



17 The freshman cohorts of CHSRI schools that opened in 2006 
tended to have slightly better test scores and less academic and social 
needs than other freshman cohorts of the other CHSRI schools. This 
change may have happened because students from these schools were 
not drawn from schools already on probation. In addition, these 
schools had an easier time recruiting a diverse pool of applicants since 
they did not have to overcome long-standing negative reputations. 

18 See Allensworth and Easton (2005) for a more detailed description 
of the Consortium’s on-track indicator. 

19 Roderick, Nagaoka, and Allensworth (2006); Allensworth and 
Easton (2007). 

20 Allensworth and Easton (2005). 

21 Roderick, Nagaoka, and Allensworth (2006). 

22 Rothstein (2004). 

23 See Allensworth and Correa (forthcoming) and Nichols and 
Berliner (2005) for discussion of how pressures to raise test scores 
negatively impacts instruction in schools. 

24 We used residual files from a two-level hierarchical model to 
identify schools who were better than expected at the p<.05. 

25 p < .05. 

26 In our earlier study of professional communities in CHSRI 
schools (Stevens [2006]), we described two different improvement 
activities in teacher groups: supportive and developmental practices. 
Supportive practices are interactions through which teachers exchange 
information and advice for addressing specific classroom tasks, 
problems, or concerns. While extremely useful for providing mutual 
support for performing everyday responsibilities, supportive practices 
will not necessarily help improve overall instruction in schools because 
they primarily take place between pairs of teachers and address 
individual concerns. In contrast, developmental practices involve 
sustained, collective work on common instructional issues and are 
thus more likely to lead to more systemic improvements across a 
group of teachers. 
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What Matters for Strong Student Outcomes? 






T he distribution of characteristics and outcomes in our sample suggests that CHSRI schools have 
been successful at developing the capacity of school staff to work together and contribute to 
school activities. Yet they have been less successful at consistently strengthening the school experience 
or course performance of freshman students. Looking across the columns in Table 2 (see page 3), we 
find that developmental practices, deep principal leadership, and strong teacher influence were the most 
common school characteristics, with each present in six schools. On the other end of the spectrum, 
strong student supports and academic press were rarely present: just three schools had strong student 
supports and no school had strong academic press. 

Going down the rows in Table 2 we also see that there was considerable variation in the number 
of characteristics schools exhibited: three schools were strong in four of the examined areas, one school 
was strong in three areas, four schools were strong in one or two areas, and two schools were not strong 
in any of the five areas. Finally, the last column in Table 2 shows that only four out of the ten schools 
had strong course performance compared to all CPS high schools serving similar students. 

Formal analysis of the distribution of characteristics and outcomes finds that there are two com- 
binations of attributes associated with strong course performance. The first, found in three schools, 
combines developmental practices, deep principal leadership, teacher influence, and strong student 
supports. The second combination, found in one school, contains three of these characteristics: de- 
velopmental practices, deep principal leadership, and teacher influence. 1 Notice in Table 2 that two 
schools contained developmental practices and deep principal leadership, while two other schools had 
strong teacher influence. Yet unlike the four schools with all three characteristics, none of these four 
schools produced strong first-year course performance. This suggests that strong principal leadership, 
developmental practices, and strong teacher influence are not sufficient individually to produce strong 
course performance; the combination of all three is both sufficient and necessary to do so. 

The importance of student supports is not as clear, however. On the one hand, since School 4 
was able to produce strong course performance without strong student supports, this factor could be 
eliminated from our consideration, leaving just the combination of deep principal leadership, develop- 
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TABLE 2 

Presence or Absence of Causal Conditions and Outcomes for CHSRI Schools 



Qualitative Indicators 


Quantitative Indicators 


Outcome 




Developmental 

Practices 


Deep Principal 
Leadership 


Teacher 

Influence 


Strong 

Student 

Supports 


Academic 

Press 


Strong Course 
Performance 


School 1 


1 


1 


1 


1 


0 


1 


School 2 


1 


1 


1 


1 


0 


1 


School 3 


1 


1 


1 


1 


0 


1 


School 4 


1 


1 


1 


0 


0 


1 


School 5 


1 


1 


0 


0 


0 


0 


School 6 


1 


1 


0 


0 


0 


0 


School 7 


0 


0 


1 


0 


0 


0 


School 8 


0 


0 


1 


0 


0 


0 


School 9 


0 


0 


0 


0 


0 


0 


School 10 


0 


0 


0 


0 


0 


0 



Note: “1 " indicates the presence of a condition or outcome; “0" indicates its absence. 



mental practices, and teacher influence to explain 
the presence of the strong course performance in 
our sample. Yet, three of the four schools with 
strong course performance also had strong student 
supports, suggesting that schools with the outcome 
tend to share this characteristic. Given this strong 
tendency in our sample and other studies showing 
the importance of personalized environments for 
students, we believe it is more likely that strong 
student supports are an important component of 
successful schools. 

Finally, these findings suggest that a high 
level of academic press is not necessary for schools 
to produce strong first-year course performance. 
As mentioned earlier, strong academic press was 



completely absent in our sample of schools. Despite 
its absence, four CHSRI schools had high stu- 
dent achievement compared to other similar high 
schools. 

It is possible, however, that we are unable to 
capture the relevance of academic press for two 
reasons. First, the overall between-school variation 
in our academic press measure is relatively low . 2 
If all schools score too similarly on the measure, 
it might not be “sensitive” enough to identify real 
differences between them. Second, if the small 
population of CHSRI schools in our sample is 
somehow exceptional, we might miss a relationship 
between academic press and course performance 
that actually exists in the general population of 
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high schools. While both of these accounts are 
plausible, additional analysis does not seem to 
support them. For instance, earlier research on the 
CHSRI initiative found statistically more positive 
differences in reports of academic press by junior 
students in CHSRI schools compared to juniors 
in other high schools using the same measure with 
approximately the same between-school variation 
(5.0 percent vs. 5.6 percent variation). 3 Thus, 
the measure can pick up the differences between 
schools if they exist, and CHSRI schools do seem 
to be doing a better job at academically challenging 
some students. In addition, we performed a sepa- 
rate analysis examining the relationship between 
schools’ average academic press and average GPAs 
and on-track rates across all high schools in CPS. In 
both cases, no statistically significant relationship 
exists. 4 In short, neither between-school variation 
nor sample size seems to explain why academic 
press is not associated with high levels of student 
achievement in our findings. 

How Do Key Characteristics 
Work Together? 

If developmental practices, deep principal leader- 
ship, and teacher influence must all be present 
in schools with strong student achievement, how 
might these attributes work together to produce 
strong outcomes? Discussions with principals and 
teachers suggest that developmental practices are 
supported both by deep principal leadership and 
teacher influence, albeit in different ways. Deep 
principal leadership acts as a catalyst for initiat- 
ing and sustaining developmental practices. 5 Yet 



as we discussed earlier, schools that engage in 
developmental practices will not necessarily have 
strong student achievement. Teachers in schools 
with strong student achievement, however, also 
described taking an active role in shaping collective 
improvement work. This influence appears to help 
create a context in which developmental practices 
are both meaningful and relevant to teachers’ 
classroom responsibilities. 

Teachers’ and principals’ descriptions of their 
improvement activities help to illustrate these 
connections. For example, Paulina High School’s 
efforts to develop grade-level performance expecta- 
tions for students show how deep principal leader- 
ship can foster developmental activities. Paulina 
High’s work to develop grade-level performance 
expectations began with the principal building 
on what his teachers learned at an action research 
workshop sponsored by the CHSRI intermediary 
organization. 6 The principal, collaborating with 
this group of teachers, asked each department to 
resubmit its end-of-year performance goals, dif- 
ferentiating them by grade levels. In addition to 
helping the school focus more on student learning, 
the rationale for developing the grade-level goals, 
according to the principal, was to 

“be very concrete with our school community in terms 
of our expectations of our school experience at each 
grade level and guide the decisions we make within 
the classroom over the course of the year. [We also 
wanted to have] a tool for us to collaborate as a school 
community to develop [a] collective vision about what 
we want for our students.” 

The principal and group of teachers looked 
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over each department’s submissions, noting pos- 
sible areas for revision and general patterns. They 
then shared their observations with the entire 
school staff. Through this discussion the school 
began to notice academic skills and outcomes that 
cut across disciplines. Taking this new information, 
the principal and a small group of teachers created 
a general set of grade-level expectations and again 
shared them with the whole school for feedback. 
After repeating this process several more times, 
the principal and the team of teachers eventually 
developed a working list of grade -level outcomes. 
As a result of these efforts, teachers in the school 
had a framework for developing their end-of-year 
student projects. As this case shows, the principal 
identified an instructional area he believed needed 
to be improved. Because he brought the problem 
to the attention of teachers who were already learn- 
ing about the topic and organized work around it, 
teachers in the school engaged in sustained, col- 
lective work to address the issue. 

While engaging in developmental practices 
is important, teachers in the four schools with 
strong student achievement also commented that 
such work needs to be supported by teacher input. 
Specifically, they pointed out that it was affirm- 
ing to share their views and shape improvement 
activities. Both teachers’ substantive contribu- 
tions and heightened feelings of self-worth made 
it easier for them to accomplish collective work 
and make it useful. For example, a teacher at one 
school described that she and her colleagues were 
able to meet their curricular improvement goals 
in part because: 



“We are valued as professional here, so when we 
have something to say about the [curriculum] map or 
we think something is not going the right way or it 
should go a different way, our professional opinion . . . [is] 
valued. So that makes all the difference in the world 
in getting that kind of stuff done.” 

Her colleague added that when teacher input 
is not respected: 

“[Curriculum mapping] just becomes another thing 
you have to do on paper instead of a meaningful 
working document that’s going to increase teaching 
and learning.” 

An improvement activity at Adams High 
School provides another example of this input. 
During December 2004, Adams’s principal began 
working with teachers to develop fall semester final 
exams. The once-a-week-exam meetings during the 
month opened the principal’s eyes to the fact that 
there was wide variation in teachers’ understanding 
of how to develop and write quality assessments. 
To address this, she and the school’s teachers spent 
the second semester working together to create 
better assessments. While engaged in this work, 
the principal also had an opportunity to attend an 
out-of-state workshop on assessments. She recalled 
being “blown away” by the session and asked the 
facilitator to come and train her school’s staff for 
two days during the summer on the creation of 
effective assessments. 

After the workshop, the school decided that 
for the 2005-06 school year they would create 
a system of interim assessments tied to college 
readiness standards because “if we are [going 
to be a] college prep [school], that means ACT 
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scores count.” Teacher teams in English, reading, 
and math began meeting with school administra- 
tors that autumn to understand the specific skills 
outlined in the college readiness standards and 
to develop assessments around them. The school 
eventually created a color-coded charting system 
to track the percentage of students in each class 
who were meeting the performance benchmarks. 
After each assessment, the charts were used in 
group discussions of students’ progress and in- 
structional strategies aimed at boosting learning. 
Adams High School’s assessments are now given 
four times a year. 

Throughout the process of creating this 
system, teachers in the school had substantial 
control over what should be included in the 
assessments, how to tie them to their own 
curricula, and how to use the collected data to 
improve students’ performance. Their influence 
was reflected in the encouragement they received 
from their principal. One teacher described the 



principal’s attitude this way: 

“ [The principal says] if you want to do it that way, you 
do it that way, and then we’ll make it happen. She’s 
saying . . . ‘Yes I trust you . . . let me know what you 
think.’” 

In sum, developmental practices seem to 
be supported by both deep principal leadership 
and teacher influence. The first helps to initiate, 
encourage, and sustain collective work in schools, 
while the latter helps make it more meaningful 
and engaging for teachers. Having both of these 
supports present in conjunction with developmen- 
tal practices was associated with strong first-year 
course performance in CHSRI schools. When only 
one support, such as deep principal leadership, 
was present with developmental practices, schools 
did not produce strong student achievement. This 
suggests that it may not be enough for teachers to 
simply engage in developmental practices. Rather, 
teachers also need to actively shape the work if it is 
going to impact student course performance. 



Endnotes 

1 Because our analysis does not privilege presence of conditions 
over their absence, technically the absence of strong academic press 
in schools is part of the combination of characteristics associated with 
strong student course performance. There is however, no theoretical 
reason to think that the absence of this condition is important for 
schools’ success and therefore we do not consider it to be relevant to 
our explanation. 

2 See Appendix B for the between-school variation of all measures. 

3 Kahne, Sporte, and de la Torre (2006). 



4 Analysis was performed using a two-level HLM. See Appendix C 
for model. 

5 This is consistent with our earlier finding about the importance of 
principal leadership for developmental practices in Stevens (2006). 

6 Action research projects are a process for teachers to systematically 
document, reflect upon, and assess selected instructional and learning 
practices (see Sagor [2000]). Intermediary organizations are non-profit 
groups that received grants to develop and provide technical assistance 
to Gates-funded small schools. 
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Interpretive Summary 






J oining the chorus of other small school researchers and reformers, this study finds that reducing 
school size does not guarantee success; only four out of ten schools in our sample produced strong 
first-year course performance compared to other schools serving similar students. Nor does small size 
ensure schools will be able to make deep changes to their environment. Several of the schools we exam- 
ined did not have strong teacher professional communities and most did not provide better personalized 
academic and social supports to their students. These findings remind us that there is an important 
difference between changing the collegiality and affective character found in schools, which small size 
seems to improve, and transforming how teachers work together and with their students. The former 
requires only that relationships become more familiar, while the latter requires schools to create new 
routines for monitoring and attending to the development of teachers and students. 

When small schools are able to change how adults work together, however, student achievement 
improves. Every school with a strong professional community, deep principal leadership, and strong 
teacher influence was able to improve students’ first-year course performance. This finding has several 
important implications. First, it suggests that teachers working collectively to improve instruction may 
be one of the most important school conditions for improving first-year performance in small schools. 
Collective improvement work on common instructional issues makes it more likely to influence 
classroom practices across larger groups of teachers. In addition, others have found that high levels of 
professional community are associated with authentic pedagogy . 1 Thus, schools that engage in profes- 
sional community activities, such as the developmental practices described in this report, would be 
more likely to raise student achievement and improve instruction. 

This collective work, however, requires robust school leadership. Deep principal leadership was 
crucial for monitoring, organizing, and sustaining collective work in our sample of schools, while 
teacher leadership helped to make collective work more relevant for the staff engaged in it. Both forms 
of leadership were necessary for schools to have strong student achievement. Reforming leadership in 
small schools, then, is not as simple as replacing principal-controlled instructional leadership with a 
primarily teacher-controlled system. Rather, leadership tasks need to be stretched across people, roles, 
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and situations in ways that become more than 
just the sum of individual contributions . 2 And it 
requires balancing individual expertise with the 
constraints imposed by official roles. Finding such 
a balance can make leadership more sustainable 
and lead to enduring improvements in instruction 
and learning . 3 

Providing students with personalized academ- 
ic and social supports also appears to be important 
for student achievement, as three of the four schools 
with this characteristic had strong student course 
performance. Yet, one school was able to produce 
strong outcomes without it. This suggests that 
while personalized student supports may facilitate 
improvement, they may not be essential for schools 
to raise student achievement. Furthermore, efforts 
to create more personalized environments may not 
lead to improvements in cases where leadership and 
professional community are absent. 

Contrary to what we expected to find, strong 
first-year course performance was not associated 
with high levels of academic press. In fact, first- 
year students in all of the CHSRI schools reported 
only average levels of academic press. Additional 
comparisons with junior CHSRI students as well as 
an analysis of academic press’ unique relationship 
with student outcomes showed that our findings 
are not the result of obvious methodological issues. 
Instead, our findings may be due to the fact that 
the transition to high school is new and challenging 
for most students and, therefore, freshmen report 
similar levels of difficulty and demands. Or in high 
performing schools, students could be receiving 
rigorous instruction but not feel pressed. In later 
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years, however, differences in reports of press might 
emerge and have a greater impact on students’ course 
performance. This explanation is partially supported 
by our earlier work showing that junior CHSRI 
students, but not freshmen, reported higher levels 
of academic press than similar students in similar 
schools . 4 Another explanation could be that aca- 
demic press may have more influence on other out- 
comes, such as dropout rates, than it does for course 
performance. While we would be hard-pressed to 
say that academic press is not important, this study 
finds little evidence demonstrating its influence on 
school-level outcomes during the freshman year. It 
does suggest, however, that more research is needed 
on how first-year students perceive and respond to 
academic press. 

Finally, this study confirms earlier Consortium 
work that shows improving student achievement 
requires schools to simultaneously develop multiple 
areas of instructional supports . 5 Strength in just one 
or two areas is not enough to produce significant 
improvements. In this case, a combination of strong 
professional community, deep principal leadership, 
and teacher influence was necessary for schools to 
produce high student achievement. Developing 
multiple areas of schools’ organizational charac- 
teristics will require intentional and concerted ef- 
forts on the part of school leaders and small school 
reformers. If such efforts lead to improved student 
achievement, it will be time well spent. 



Endnotes 

1 Louis, Kruse, and Marks (1996). 

2 Spillane, Halverson, and Diamond (2004). 

3 Hargreaves and Fink (2003). 

4 Kahne, Sporte, de la Torre (2006). 

5 Sebring et al. (2006). 
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Appendix A: Description of the Sample 






Teacher and Student Surveys 

D ata used in our survey analysis were taken from our 2005 biannual survey. Students, teachers, 
and principals in all 103 non-alternative high schools were invited to participate in our survey. 
Approximately 4,150 teachers in 87 schools responded, as did approximately 35,600 students in grades 
9 and 10, and 10,600 students in grade 11. Only schools with at least a 50 percent survey response rate 
were included in our sample. We also did not include eight CPS achievement academies. Achievement 
academies are separate schools within a larger high school for students who did not meet the promotion 
criteria to attend a high school and are too old to remain in elementary school. Since their population, 
curriculum, and supports are different than what first-time freshmen experience, they are not part of 
our sample. 

TABLE A1 

Consortium on Chicago School Research Overall Survey Response Rates 





Number 
of Surveyed 
Schools 


Number 
of Responding 
Schools 


Number 
of Surveyed 
Individuals 


Number 
of Responding 
Individuals 


Surveyed Schools: 
Individual 
Response Rate 


Responding Schools: 
Individual 
Response Rate 


Teacher 

Survey 


103 


87 


7,635 


4,142 


54% 


59% 


Ninth- and 
Tenth-Grade 
Student Survey 


101 


87 


60,615 


35,608 


59% 


64% 



TABLE A2 

Analytic Sample* 



Number of Schools with Response Rates of 
50% or More 


Number of Individuals in These Schools That 
Responded 




CHSRI Sample 


Other Schools 


CHSRI Sample 


Other Schools 


Teacher 

Survey 


10 


42 


220 


3,237 


First-Time 
Ninth-Grade 
Student Survey 


10 


61 


753 


15,544 



* Number of individuals in the sample varied slightly depending on the measure being analyzed. See Appendix B. 
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Student Outcomes 

We analyzed outcome data from all CPS high schools except alternative schools and the achievement 
academies described above. 



TABLE A3 

Students and Schools Used in Analysis of Raw GPA and On-Track (to Graduate) Rates 





Number of Students 


Number of Schools 




CHSRI Sample 


Other Schools 


CHSRI Sample 


Other Schools 


2005-06 


1,419 


25,954 


10 


67 
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Appendix B: Survey Measures and Factors 






W e combined multiple items from our teacher and student surveys into measures using Rasch 
analysis . 1 Each measure is on a continuous, linear scale that can be used for statistical pro- 
cedures. Survey items are used to define measures based on the relative probability of a respondent 
choosing each category on each item. A fit statistic is used to omit or include items into measures. The 
internal consistency of scale items is also evaluated using the person reliability statistic. Individuals are 
then placed on measure scales based on their particular responses to items in the measure. Placement 
on measure indicates the “amount” of a characteristic or skill that an individual possesses. 

For our indicator of personalized student supports, we combined six measures into a single factor 
using principal component analysis. This technique examines the relationship between several observed 
variables to identify underlying factors that can be represented with a smaller number of variables. 



TABLE B1 

Teacher and Student Measures 



Measure 



Teacher Influence 



3,258 



Individual 

Reliability 



0.84 



Between-School 

Reliability 



0.89 



Between-School 

Variation 



21.9% 



Description: Measures the extent of teachers’ involvement in school decision making. It assesses teachers’ influence on the selection of 
instructional materials, setting of school policy, in-service program planning, spending of discretionary funds, and hiring of professional 
staff. 



Academic Press 



12,481 



0.76 



0.75 



5.0% 



Description: Students’ views of their teachers’ efforts to push students to higher levels of academic performance. Students also report on 
the degree to which they find their classes to be challenging. 



Student Support Factor 



12,453 



0.80* 



0.89 



9.9% 



Description: A factor combining the following six measures of supports students receive: school-wide future orientation, sense of 
belonging, peer support for academic achievement, classroom personalism, student-teacher trust, and teacher support. 

* Cronbach Alpha 



Appendix B Endnote 

1 Wright and Masters (1987). 
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Appendix C: Models Used in This Report 



Teacher Measure/Teacher Influence 

W e used a three-level hierarchal liner model (HLM) to examine whether teachers at CHSRI schools 
had higher than expected levels of teacher influence, as measured by our 2005 teacher survey. 1 
The first level adjusted for measurement error produced by the Rasch analysis. At level two we modeled 
individual teachers’ “real” scores by controlling for the following background characteristics: gender, 
race, years of experience, level of education, whether the respondent entered the profession through an 
alternative process, and whether the respondent was a classroom teacher. We fixed the slopes for these 
variables at level three so that relationship between each variable and the outcome measure was held 
constant. The intercept in the level two equation can be interpreted as the school mean adjusted for 
individual background characteristics. At level three we adjusted for differences in the following school 
characteristics: incoming eighth-grade achievement level of the student body, adjusted for current grade; 
aggregate socioeconomic status; and whether the school was at least 70 percent African American. All 
variables were grand mean centered. 

Using the residual file produced by the HLM analysis, we divided the empirical Bayes residual for 
the level two intercept by the square root of the posterior variance. If the result was greater than 1.96, 
the school was flagged as having strong teacher influence. 



Level One 



Measure ■, 

jk 



S-, 



1 




where e^ ~ N(0,1), s^ k is the standard error estimated from the Rasch analysis for teacher^' in school 
k and n jk is the teacher’s “true score.” 



Level Two 

7l jk = Pok + Pik( Female )jk + ^2k ( Latin °)jk + P3k( WFlte V + p4k( Asian )jk + P 5 k( Native American )jk + P6k 
(Other)j k + [3 7k (Classroom Teacher)j k + (3 8k (Bachelor’s Degree)j k + (3 gk (Doctorate Degree)j k + P 10k (3 or 

less years teaching)^ (3 nk (4to 10 years teaching) j k + (3 pk (Alternative Certification) j k + P 13k (No Gender 
Data)j k + P l4k (No Race Data) j k + P 15k (No Data on Experience) j k + (3 ]6k (No Data on Education Degree) j k + 
Pi 7 k (N° Classroom Teacher Information) j k + (3 lgk (No Alternative Certification Information) j k + r k 
A description of these variables can be found in Table Cl. 
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Level Three 

(3 0k = y 00 + Y 01 (Mean Eighth-Grade Achievement) k + Y 02 (Mean Concentration of Poverty) k + Y 03 
(70% or more African American) k + u k 

P P k = T p0 , for p = 1 to 18 

A description of these variables can be found in Table C2. 

Student Measures — Personalized Student Supports, Academic Press 

To examine whether students in CHSRI schools experience better-than-expected responses on our per- 
sonalized student supports factor we used a two-level hierarchical model. At level one we adjusted for 
the following individual student background characteristics: gender, race, socioeconomic status, prior 
academic achievement as measured on the eighth-grade Iowa Tests of Basic Skills (ITBS) reading test, 
whether a student was receiving special education services, and whether a student was old for grade. All 
slopes were held constant across schools. At level two we adjusted for the same school characteristics as 
we did in our teacher analysis of teacher influence. All variables were grand mean centered. 

For academic press, we used a three level model. Level one adjusted for measurement error in 
each person-measure. In level two we controlled for the same set of student characteristics described 
in our model of personalized student supports and in level three we adjusted for the same school-level 
characteristics discussed in the previous models. 

We again used residual files from our HLM analysis to calculate whether schools were better than 
expected on each measure. Schools were identified as having strong personalized student supports or 
academic press if the result of dividing its empirical Bayes residual by the square root of the posterior 
variance was greater than 1.96. 

Student Supports 
Level One 

Y jk = Pok + P lk (Female) jk + P 2 k( Latin °)jk + p 3 k( WFite )jk + f^ 4 k (Asian) jk+ P 5k (Native American) 
jk+ P 6k (Concentration of Poverty) - k + (3 7k (Special Education) j k + (3 gk (Eighth- Grade Achievement 
Reading)j k + (3 gk (Months Old Began High School)- k + P 10k (No Eighth-Grade Score)- k + (3 nk 
(No Economic Data)j k + r k 

A description of these variables can be found in Table Cl. 
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Level Two 

Pok = Too + Yoi (Mean Eighth-Grade Achievement) k + y ()2 (Mean Concentration of Poverty) k + y 03 
(70% or more African American) k + u k 

P P k = Y p o > for P = 1 to 10 



Academic Press 

Level One 



Measure 



jk 



S jk 



1 




where ej k ~ N(0,1), s^ k is the standard error estimated for the Rasch analysis for student^’ in school k 

and 7C-, is the student’s “true score.” 

J k 



Level Two 



10 



% - Pok + Ppk X pjk + r jk 



10 



where X P pk X p j k represents the ten variables and coefficients from the level one variables in 



P=i 



the student supports model. 



Level Three 

Pok = Yoo + y 0 i (Mean Eighth-Grade Achievement) k + y 02 (Mean Concentration of Poverty) k + y 03 
(70% or more African American) k + u k 

P P k = Y p o > f°r p = 1 to 10 



Student Outcomes — Residuals of Average GPA and On-Track (to Graduate) Rates; 
Relationship between Academic Press and Student Outcomes 

To identify schools with better-than-expected average GPAs and on-track (to graduate) rates, we used 
two-level hierarchical models. At level one we controlled for gender, race, socioeconomic status, prior 
academic achievement as measured on the eighth-grade Iowa Tests of Basic Skills (ITBS) reading test, 
whether a student was receiving special education services, and whether a student was old for grade. 
All slopes were held constant across schools. 
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We adjusted for school characteristics by controlling for the incoming eighth-grade achievement 
level of the student body, adjusted for current grade; aggregate socioeconomic status; and whether the 
school was at least 70 percent African American. 

As with our other models, we used residual files from our HLM analysis to calculate whether 
schools were better than expected on each outcome. Schools were identified as having strong average 
GPAs and on-track (to graduate) rates if the results of dividing empirical Bayes residuals by the square 
root of the posterior variance were greater than 1.96. 

To examine the relationship between school-level academic press and average GPA and on-track 
(to graduate) rates, we used a two-level hierarchical model that controlled for student-level character- 
istics at level one. All slopes were held constant across schools. At level two we included a variable for 
schools’ average reports of academic press. We adjusted for school characteristics by controlling for the 
incoming eighth-grade achievement level of the student body, adjusted for current grade; aggregate 
socioeconomic status; and whether the school was at least 70 percent African American. 

Average GPA and On-Track (to Graduate) Residuals 

Level One 

Tj = Pok + Pik ( Female )jk + p2k( Latin °)jk + ^ 3 k (White) jk + P 4k (Asian) j k + P 5k (Native American) jk + (3 6k 
(Concentration of Poverty) j k + (3 7k (Special Education) - k + (3 gk (Eighth-Grade Achievement Reading) 
jk+ (3 9k (Months Old Began High School)j k + (3 10k (No Eighth-Grade Score)j k + (3 nk (No Economic 

Data )jk + r jk 

Level Two 

P 0 k = Y 00 + Y 01 (Mean Eighth-Grade Achievement) k + y Q1 (Mean Concentration of Poverty) k + y 03 
(70% or more African American) k + u k 

P P k = T p0 , for p = 1 to 11 



For On-Track 

Yj I <Pij ~ P(h (Pij) and T| ;j = log [1 - cpjj] 



For GPA 

ty - GPA.. 

'ii i) 
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Relationship between Academic Press and Student Outcomes: 

Level Two 

11 

^jk = Pok + ^ Ppk^pjO r jk’ 

11 

where + X (3 pk X p ^ k represents the eleven variables and coefficients from the level one variables 
in the outcomes residuals model. 

Level Two 

(3 0k = y 00 + y 01 (Mean Academic Press) + y 01 (Mean Eighth-Grade Achievement) k + y 0 -, 

(Mean Concentration of Poverty) k + y 03 (70% or more African American) k + u k 

P P k = T p0 - for P = 1 to n - 
For On-Track 

cp.. 

Tij I % - P0. <Pij) and Tly = log [- — ] 

For GPA 

tv. - GPA.. 

*u ‘I 



Appendix C Endnote 

1 Raudenbush and Bryk (2002). 
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Description of the Variables Used in Models 



TABLE Cl 



Individual-Level Variables Used in Analyses 



Where Used Description 



Teacher Survey 
Measure Analysis 



Gender was indicated by a dummy variable. 

Race/Ethnicity was indicated by a set of dummy variables. Categories included “white, non-Hispanic,” “Hispanic,” 
“American Indian,” “Asian,” and “other” (which combined “biracial/multi-ethnic” and “other”). 

Classroom Teachers were indicated by a dummy variable, distinguishing teachers who did not have a specific 
classroom responsibility from those who did. 

Education Level was indicated by a set of dummy variables. Categories included “Bachelor’s degree,” “Master’s 
degree” (which combined “Master’s degree” with “Master’s degree plus 15 credits or more”), and “Doctorate.” 

Teaching Experience was indicated by a set of dummy variables. The omitted category was teachers who 
indicated they had 11 to 15 or more than 15 years teaching experience. 

Entering Teaching through an Alternative Certification Program was indicated by a dummy variable, 
distinguishing such teachers from those who entered the profession through other traditional programs. 



Missing Values were imputed at the sample mean, and a dummy variable was assigned. 



All Analyses of 
Student Surveys 
and Outcomes 



Gender was indicated by a dummy variable. 

Race/Ethnicity was indicated by a set of dummy variables. Categories included “white, non-Hispanic,” “Hispanic,” 
“American Indian,” and “Asian.” 



Socioeconomic Status was based on a measure of Concentration of Poverty. This variable is derived from the 
2000 U.S. Census information on the census block group in which students lived. Students’ home addresses 
were linked to a particular block group within the city, which could then be linked to census data on the economic 
conditions of the student’s neighborhood. Two indicators were used to construct the variable: (1) log of the 
percentage of families above the poverty line, and (2) log of the percentage of men employed in the block group. 

Receiving Special Education Services was based on administrative records. 

Student’s Elementary Achievement was based on student reading scores on the Iowa Tests of Basic Skills (ITBS) 
taken in eighth grade. The analyses of student outcomes included linear and quadratic terms for this variable. 

Months Old Began High School gives the number of months older than 14 years and eight months that a student 
was as of September 1 of the academic year. Students older than this age should have started high school with the 
previous cohort if they were to follow school-system guidelines. 

Missing Values. If students were missing values on socioeconomic status or their eighth-grade achievement, they 
were given values at the sample mean and assigned a dummy variable. 



TABLE C2 

School-Level Variables Used in Analyses 



Where Used 


Description 


All Analyses 


School-Level Achievement was computed by averaging the eighth-grade ITBS reading scores of all current 
members of a school’s student body, adjusting for current grade level. The analyses of student outcomes included 
linear and quadratic terms for this variable. 




School-Level Socioeconomic Status. A description of how this variable was created at the individual level can be 
found in Table Cl. The school-level indicator was an average of the concentration of poverty of all students in the 
school based on the census block in which each lived. 




Racial Composition of the School. A dummy variable was included for schools with a student body that was at 
least 70 percent African American. 
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Appendix D: Qualitative Methodology 



T he qualitative data were first organized according to several categories of research questions in- 
cluding, but not limited to, academic improvement activities pursued over the course of the year, 
principal and teachers’ role in these activities, and the specific tasks each improvement activity involved. 
We then used a process of open coding to generate several analytic and descriptive themes. Once themes 
relevant to the questions pursued in this report were identified, we engaged in an iterative process of 
focused coding where themes were further elaborated and more abstract concepts identified . 1 We then 
put data into cross-case displays to explore possible patterns and variations across schools . 2 

To identify school characteristics that produce strong student achievement, we used qualitative 
comparative analysis (QCA), which is especially suited to examining data sets with a small to medium 
number of cases . 3 For each case in our sample, we identified whether it exhibited the outcome (strong 
course performance) and each causal condition (school characteristics) . We then constructed a truth- 
table, or data matrix, with every possible combination of causal conditions. Using Boolean algebra, we 
reduced the table to identify combinations of factors leading to our outcome of interest. 

Rather than identifying the net effects of variables, QCA is used to explore the ways in which 
specified conditions combine with one another and work together to produce particular outcomes or 
events. It also allows for the possibility that there are multiple paths to outcomes of interest. Yet, it 
simplifies analysis by eliminating irrelevant factors. When two combinations leading to an outcome 
are the same except for one condition, that condition is considered irrelevant and can be dropped for 
the analysis. This allows the two combinations to be combined into one, simplifying the analysis. 



Appendix D Endnotes 

1 Emerson, Fretz, and Shaw (1995). 

2 Miles and Huberman ( 1 994) . 

3 Ragin (1987). 
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W. David Stevens 

W. David Stevens is a Senior Research Analyst for the Consortium on Chicago School Research at 
the University of Chicago (CCSR). His research focuses on high school reform and instructional 
development. He was the lead author on an earlier CHSRI report entitled Professional communities 
and instructional improvement practices: A study of small high schools in Chicago. He recently began 
a three-year mixed-methods study of the transition to high school, which will follow a cohort of 
students from eighth grade into their second year of high school. This study looks at the ways in 
which students perceive the challenges of high school, the school practices that can foster successful 
freshman-year performance, and those that can hinder students. Stevens holds a Ph.D. in Sociology 
from Northwestern University. 

This report reflects the interpretation of the author. Although the Consortium ’s Steering Committee provided 
technical advice and reviewed earlier versions, no formal endorsement by these individuals, organizations, or the 
full Consortium should be assumed. 



This report was produced by the Consortium's publications and communications staff. 

Editing, project management, and graphic design by Office of Publications & Creative Services, The University of Chicago. 
Photo by Cynthia Howe. 
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